Evaluating Multilinguality in Large Language Models

Lecture presented by François Yvon (LISN-CNRS).

The Large Language Models introduced in the recent years have been found extremely helpful to advance the state-of-the-art in many Natural Language Applications, notably due to their ability to compute numerical, high-dimensional, representations of linguistic units such as words or sentences. Multilingual language models go one step further and add the ability to handle multiple languages, sometimes even multiple scripts, with just one single model. In this presentation, I will discuss multilingual language models at length, how they are typically learned and used, with a focus on the measurement of their multilingual abilities. The main question I will thus try to answer is "what does it mean for a multilingual model X to cover language Y ?".

François Yvon is a senior CNRS researcher at the LISN (formerly LIMSI) laboratory of Université Paris Saclay in Orsay, France. F. Yvon has been leading activities in Machine Translation at LISN for about 15 years, resulting in more than one hundred scientific publications on all aspects related to the development and evaluation of multilingual language processing technologies, from word and sentence alignment to translation modelling and evaluation, including recent work on multi-domain adaptation in MT and on cross-lingual transfert learning issues. He has acted as coordinator or PI in multiple past national and international projects in MT such as Quaero or H2020/QT21 and has supervised more than 15 PhDs on MT related topics. He was recently involved in the evaluation activities of the large "Big Science" collaboration. He is a board member of the European chapter of the Association for Computational Linguistics, of the MetaNet network, and has recently contributed as an expert on linguistic technologies for the French language to several European projects (European Language Resource Collection, ELE – European Language Equality, ELG – European Language Grid).

Mots clés : ia informatique jsalt language models measurement multilingual abilities nlp workshop